Depth sensor and method of operating the same

ABSTRACT

A method of operating a depth sensor includes generating a first photo gate signal and second through fourth photo gate signals respectively having 90-, 180- and 270-degree phase differences from the first photo gate signal, applying the first photo gate signal and the third photo gate signal to a first row of a pixel array and the second photo gate signal and the fourth photo gate signal to a second row adjacent to the first row in a first frame using a first clock signal, and applying the first photo gate signal and the third photo gate signal to a first column of the pixel array and the second photo gate signal and the fourth photo gate signal to a second column adjacent to the first column in a second frame using a second clock signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 14/150,361, filed Jan. 8, 2014 which claims priority to KoreanPatent Application No. 10-2013-0137176 filed on Nov. 12, 2013 in theKorean Intellectual Property Office (KIPO), the contents of which arehereby incorporated by reference herein in their entireties.

BACKGROUND

Embodiments of the inventive concept relate to a depth sensor and amethod of operating the same, and more particularly, to a depth sensorfor acquiring image data with noise minimized and a method of operatingthe same.

With the rapid increase of demand for smart phones, there has been a lotof development of image sensors. Image sensors include a plurality ofpixels converting photons in a predetermined spectrum band intoelectrons.

Information about the depth between an object and an image sensor aswell as information about color may be necessary to obtain athree-dimensional (3D) image. Methods of obtaining the information aboutthe depth between the object and the image sensor may be divided intotwo types: active methods and passive methods.

A time-of-flight (TOF) method and a triangulation method arerepresentative active methods. In the TOF method, modulated light isemitted to an object, light reflected from the object is sensed, and thedepth is calculated from a phase change. In the triangulation method,the position of light emitted by a laser or the like in a certaindistance from a sensor and the position of reflected light are sensedand the depth is calculated using triangulation. In the passive methods,the depth is calculated using only image information without emittinglight. A stereo camera may be representative of the passive methods.

SUMMARY

According to some embodiments of the inventive concept, there isprovided a method of operating a depth sensor. The method includesgenerating a first photo gate signal and second through fourth photogate signals respectively having 90-, 180- and 270-degree phasedifferences from the first photo gate signal; applying the first photogate signal and the third photo gate signal to a first row of a pixelarray and the second photo gate signal and the fourth photo gate signalto a second row adjacent to the first row in a first frame using a firstclock signal; and applying the first photo gate signal and the thirdphoto gate signal to a first column of the pixel array and the secondphoto gate signal and the fourth photo gate signal to a second columnadjacent to the first column in a second frame using a second clocksignal. A frequency of the first clock signal may be different from afrequency of the second clock signal.

The method may further include interpolating pixel signals output fromrespective pixels included in the pixel array and generating image databased on interpolated pixel signals.

The interpolating the pixel signals may include interpolating a pixelsignal of a first pixel in the first row based on a pixel signal of atleast one second pixel in the second row in the first frame andinterpolating the pixel signal of the second pixel in the second rowbased on a pixel signal of at least one first pixel in the first row inthe first frame.

The interpolating the pixel signal of the first pixel in the first rowbased on the pixel signal of the at least one second pixel in the secondrow may include applying a weight corresponding to an offset of each ofthe at least one second pixel to the pixel signal of each at least onesecond pixel. The interpolating the pixel signal of the second pixel inthe second row based on the pixel signal of the at least one first pixelin the first row may include applying a weight corresponding to anoffset of each of the at least one first pixel to the pixel signal ofeach at least one first pixel.

The weight may be calculated based on a Gaussian function.

The interpolating the pixel signals may further include interpolatingthe pixel signal of the first pixel in the first column based on thepixel signal of the at least one second pixel in the second column inthe second frame and interpolating the pixel signal of the second pixelin the second column based on the pixel signal of the at least one firstpixel in the first column in the second frame.

The interpolating the pixel signal of the first pixel in the firstcolumn based on the pixel signal of the at least one second pixel in thesecond column may include applying a weight corresponding to an offsetof each of the at least one second pixel to the pixel signal of each atleast one second pixel. The interpolating the pixel signal of the secondpixel in the second column based on the pixel signal of the at least onefirst pixel in the first column may include applying a weightcorresponding to an offset of each of the at least one first pixel tothe pixel signal of each at least one first pixel.

The weight may be calculated based on a Gaussian function.

According to other embodiments of the inventive concept, there isprovided a depth sensor including a photo gate signal generatorconfigured to generate a first photo gate signal and second throughfourth photo gate signals respectively having 90-, 180- and 270-degreephase differences from the first photo gate signal; and a row and columnselector configured to apply the first through fourth photo gate signalsto rows or columns of a pixel array. The row and column selector appliesthe first photo gate signal and the third photo gate signal to a firstrow of the pixel array and the second photo gate signal and the fourthphoto gate signal to a second row adjacent to the first row in a firstframe using a first clock signal and applies the first photo gate signaland the third photo gate signal to a first column of the pixel array andthe second photo gate signal and the fourth photo gate signal to asecond column adjacent to the first column in a second frame using asecond clock signal. A frequency of the first clock signal may bedifferent from a frequency of the second clock signal.

The depth sensor may further include an image signal processorconfigured to interpolate pixel signals output from respective pixelsincluded in the pixel array and to generate image data based oninterpolated pixel signals.

The image signal processor may interpolate a pixel signal of a firstpixel in the first row based on a pixel signal of at least one secondpixel in the second row in the first frame and may interpolate the pixelsignal of the second pixel in the second row based on a pixel signal ofat least one first pixel in the first row in the first frame.

The image signal processor may apply a weight corresponding to an offsetof each of the at least one second pixel to the pixel signal of each atleast one second pixel when interpolating the pixel signal of the firstpixel in the first row based on the pixel signal of the at least onesecond pixel in the second row and may apply a weight corresponding toan offset of each of the at least one first pixel to the pixel signal ofeach at least one first pixel when interpolating the pixel signal of thesecond pixel in the second row based on the pixel signal of the at leastone first pixel in the first row.

The image signal processor may interpolate the pixel signal of the firstpixel in the first column based on the pixel signal of the at least onesecond pixel in the second column in the second frame and mayinterpolate the pixel signal of the second pixel in the second columnbased on the pixel signal of the at least one first pixel in the firstcolumn in the second frame.

The image signal processor may apply a weight corresponding to an offsetof each of the at least one second pixel to the pixel signal of each atleast one second pixel when interpolating the pixel signal of the firstpixel in the first column based on the pixel signal of the at least onesecond pixel in the second column and may apply a weight correspondingto an offset of each of the at least one first pixel to the pixel signalof each at least one first pixel when interpolating the pixel signal ofthe second pixel in the second column based on the pixel signal of theat least one first pixel in the first column.

The weight may be calculated based on a Gaussian function.

According to some embodiments, a method of operating a depth sensor mayinclude interpolating pixel signals output from pixels of first andsecond adjacent rows of a pixel array in a first frame and interpolatingpixel signals output from first and second adjacent columns of the pixelarray in a second frame. The pixel signals of the first frame may begenerated using a first clock signal and the pixel signals of the secondframe may be generated using a second clock signal different than thefirst clock signal. The method may also include generating image databased on the interpolated pixel signals.

The pixel signals of the first row may be generated by a first pair ofphoto gate signals out of phase with each other and the pixel signals ofthe second row are generated by a second pair of photo gate signals outof phase with each other. The pixel signals of the first column may begenerated by the first pair of photo gate signals and the pixel signalsof the second column may be generated by the second pair of photo gatesignals.

The pixel signals of the first frame and the pixel signals of the secondframe may each be generated by first, second, third and fourth photogate signals. The second, third and fourth photo gate signals mayrespectively have 90-, 180- and 270-degree phase differences from thefirst photo gate signal. The first and third photo gate signals may beapplied to the first row and first column and the second and fourthphoto gate signals may be applied to the second row and second column.

The method may further include interpolating a pixel signal of a firstpixel in the first row based on a pixel signal of a second pixel in thesecond row, interpolating the pixel signal of the second pixel in thesecond row based on a pixel signal of the first pixel in the first row,interpolating a pixel signal of a first pixel in the first column basedon a pixel signal of a second pixel in the second column andinterpolating the pixel signal of the second pixel in the second columnbased on the pixel signal of the first pixel in the first column.

The method may also include applying a weight corresponding to areflected signal offset of the second pixel to the pixel signal of thesecond pixel, applying a weight corresponding to a reflected signaloffset of the first pixel to the pixel signal of the first pixel,applying a weight corresponding to a reflected signal offset of thesecond pixel to the pixel signal of the second pixel and/or applying aweight corresponding to a reflected signal offset of the first pixel tothe pixel signal of the first pixel.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the inventive conceptwill become more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings in which:

FIG. 1 is a block diagram of a depth sensor according to someembodiments of the inventive concept;

FIG. 2 is a detailed block diagram of a photo gate controllerillustrated in FIG. 1;

FIG. 3 is a diagram for explaining the operation of the photo gatecontroller illustrated in FIG. 1 during a first frame;

FIG. 4 is a diagram for explaining the operation of the photo gatecontroller illustrated in FIG. 1 during a second frame;

FIG. 5 is a circuit diagram of a 2-tap depth pixel in a pixel arrayillustrated in FIG. 1;

FIG. 6 is a timing chart showing the operation of the 2-tap depth pixelillustrated in FIG. 5;

FIG. 7 is a diagram for explaining a method of interpolating pixelsignals output from the pixel array illustrated in FIG. 1 using a pseudo4-tap scheme according to some embodiments of the inventive concept;

FIG. 8 is a diagram illustrating reflected optical signals input to someof pixels illustrated in FIG. 7;

FIGS. 9A and 9B are diagrams illustrating a weight function used ininterpolating pixel signals;

FIG. 10 is a flowchart of a method of operating the depth sensorillustrated in FIG. 1 according to some embodiments of the inventiveconcept; and

FIG. 11 is a detailed flowchart of an operation of interpolating pixelsignals in the method illustrated in FIG. 10.

DETAILED DESCRIPTION

The inventive concept now will be described more fully hereinafter withreference to the accompanying drawings, in which embodiments of theinvention are shown. This invention may, however, be embodied in manydifferent forms and should not be construed as limited to theembodiments set forth herein. Rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. In thedrawings, the size and relative sizes of layers and regions may beexaggerated for clarity. Like numbers refer to like elements throughout.

It will be understood that when an element is referred to as being“connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element or intervening elements may bepresent. In contrast, when an element is referred to as being “directlyconnected” or “directly coupled” to another element, there are nointervening elements present. As used herein, the term “and/or” includesany and all combinations of one or more of the associated listed itemsand may be abbreviated as “/”.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first signal could be termed asecond signal, and, similarly, a second signal could be termed a firstsignal without departing from the teachings of the disclosure.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” or “includes” and/or “including” when used in thisspecification, specify the presence of stated features, regions,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,regions, integers, steps, operations, elements, components, and/orgroups thereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and/orthe present application, and will not be interpreted in an idealized oroverly formal sense unless expressly so defined herein.

Since depth information may include noise due to various factors, it isdesired to minimize the noise to acquire accurate information. FIG. 1 isa block diagram of a depth sensor 10 according to some embodiments ofthe inventive concept. FIG. 2 is a detailed block diagram of a photogate (PG) controller 28 illustrated in FIG. 1. FIG. 3 is a diagram forexplaining the operation of the PG controller 28 illustrated in FIG. 1during a first frame. FIG. 4 is a diagram for explaining the operationof the PG controller 28 illustrated in FIG. 1 during a second frame.FIG. 5 is a circuit diagram of a 2-tap depth pixel 23 in a pixel array22 illustrated in FIG. 1. FIG. 6 is a timing chart showing the operationof the 2-tap depth pixel 23 illustrated in FIG. 5.

Referring to FIGS. 1 through 6, the depth sensor 10 that can measure adistance or a depth using a time of flight (TOF) principle includes asemiconductor chip 20 including a pixel array 22 in which plural depthpixels (i.e., detectors or sensors) 23 are arranged, a light source 32,and a lens module 34. The depth pixels 23 may be implemented in 2-taparchitecture illustrated in FIG. 5. The 2-tap architecture enables imagedata to be obtained with less number of sampling operations than 1-taparchitecture and provides a higher fill factor than 4-tap architecture.

The depth sensor 10 may be implemented as part of an image sensor (notshown) that generates a three-dimensional (3D) image using a color imageand a depth image.

The depth pixels 23 implemented in the pixel array 22 in two dimensionsinclude transfer gates, e.g., TX1 and TX2 in FIG. 5. The depth pixels 23also include a plurality of transistors for signal processing.

A row decoder 24 selects one row from among a plurality of rows inresponse to a row address output from a timing controller 26. Here, arow is a set of depth pixels arranged in a horizontal direction in thepixel array 22.

The PG controller 28 may generate a plurality of PG signals PGS1 throughPGS4 and provide them to the pixel array 22 under the control of thetiming controller 26. As shown in FIG. 2, the PG controller 28 mayinclude a clock selector 51, a frame index 52, a PG signal generator 53,and a row and column selector 54.

The clock selector 51 may select and output one of a first clock signalCLK1 and a second clock signal CLK2, which are output from the timingcontroller 26, according to the control of the frame index 52. The firstand second clock signals CLK1 and CLK2 have a first frequency and asecond frequency, respectively. The first frequency may be differentfrom the second frequency. For instance, the clock selector 51 mayselect and output the first clock signal CLK1 during an odd-numberedframe and may select and output the second clock signal CLK2 during aneven-numbered frame. In other embodiments, the clock selector 51 mayoutput a clock signal (either of the first and second clock signals CLK1and CLK2) having the same frequency in all frames.

The frame index 52 may receive a clock signal MLS for driving the lightsource 32 from a light source driver 30. The frame index 52 maydetermine whether a current frame is an odd-numbered frame or aneven-numbered frame according to the clock signal MLS. The frame index52 may control the clock selector 51 and the row and column selector 54according to the determination result.

The PG signal generator 53 may receive one of the first and second clocksignals CLK1 and CLK2 and generate the first PG signals PGS1 and secondthrough fourth PG signals PGS2 through PGS4 which respectively have 90-,180-, and 270-degree phase differences from the first PG signal PGS1. Inother words, the PG signal generator 53 generates the first PG signalsPGS1 having the same phase as the clock signal MLS and the third PGsignal PGS3 having a 180-degree phase difference from the clock signalMLS. Also, the PG signal generator 53 generates the second PG signalsPGS2 having a 90-degree phase difference from the clock signal MLS andthe fourth PG signal PGS4 having a 270-degree phase difference from theclock signal MLS. The frequency of the first through fourth PG signalsPGS1 through PGS4 generated during a first frame may be different fromor the same as that generated during a second frame.

The row and column selector 54 may apply the first through fourth PGsignals PGS1 through PGS4 to rows or columns in the pixel array 22according to the control of the frame index 52.

FIGS. 3 and 4 show methods of the row and column selector 54 applyingthe PG signals PGS1 through PGS4 to part of the pixel array 22. The partof the pixel array 22 includes 16 depth pixels 23 arranged in a matrixwith four rows and four columns. For clarity of the description, onlythe part of the pixel array 22, which includes four rows and fourcolumns, is described, but all rows and columns included in the pixelarray 22 may operate in the same manner and the numbers of rows andcolumns are not limited.

FIG. 3 shows a connection state 22-1 between the row and column selector54 and the pixel array 22 during a first frame (or an odd-numberedframe). A frame is a time taken for a cycle of photocharge integration,pixel signal generation, and read-out to be completed with respect tothe entire pixel array 22.

During the first frame, the row and column selector 54 applies the samePG signal to each of the rows. In other words, during the first frame,the row and column selector 54 may apply the first PG signals PGS1 andthe third PG signal PGS3 to odd-numbered rows of the pixel array 22 andmay apply the second PG signals PGS2 and the fourth PG signal PGS4 toeven-numbered rows of the pixel array 22.

FIG. 4 shows a connection state 22-2 between the row and column selector54 and the pixel array 22 during a second frame (or an even-numberedframe). During the second frame, the row and column selector 54 appliesthe same PG signal to each of the columns. In other words, during thesecond frame, the row and column selector 54 may apply the first PGsignals PGS1 and the third PG signal PGS3 to odd-numbered columns of thepixel array 22 and may apply the second PG signals PGS2 and the fourthPG signal PGS4 to even-numbered columns of the pixel array 22.

The row and column selector 54 may include a plurality of switchingunits as many as the number of rows or columns to switch between theconnection states 22-1 and 22-2 respectively illustrated in FIGS. 3 and4 whenever a frame changes.

The PG signal generator 53 generates the PG signals PGS1 through PGS4based on the first clock signal CLK1 during the first frame, so thatimage data based on pixel signals generated by the depth pixels 23 has amaximum measurement distance depending on the frequency of the firstclock signal CLK1. For instance, when the frequency of the first clocksignal CLK1 is 20 MHz, the maximum measurement distance is set to 7.5 m.

The PG signal generator 53 generates the PG signals PGS1 through PGS4based on the second clock signal CLK2 during the second frame, so thatimage data based on pixel signals generated by the depth pixels 23 has amaximum measurement distance depending on the frequency of the secondclock signal CLK2. For instance, when the frequency of the second clocksignal CLK2 is 30 MHz, the maximum measurement distance is set to 5 m.

When the image data based on the first clock signal CLK1 having afrequency of 20 MHz giving a maximum measurement distance of 7.5 mduring the first frame is mixed with image data based on the secondclock signal CLK2 having a frequency of 30 MHz giving a maximummeasurement distance of 5 m during the second frame, a maximum distancerepresented by two frames is 30 m, i.e., a common multiple of the twomaximum measurement distances.

When the depth pixels 23 have the 2-tap architecture, two frames arebasically required to generate a depth image corresponding to the pixelarray 22. However, when pseudo 4-tap scheme is used, a depth image canbe generated with only one frame. In order to generate complete depthinformation using a single depth pixel 23, a pixel signal generated byfour PG signals respectively having different phases is required. Sincethe depth pixel 23 has the 2-tap architecture for a single frame, itgenerates a pixel signal from two PG signals respectively havingdifferent phases. According to the pseudo 4-tap scheme, a result ofinterpolating a pixel signal generated by another depth pixel 23 that isadjacent to the current depth pixel 23 and receives two other PG signalsrespectively having different phases is used as a pixel signal generatedby the current depth pixel 23 from the two other PG signals.

When image data is formed with only pixel signals generated for thefirst frame using the pseudo 4-tap scheme, spatial resolution decreasesin a vertical direction. This is because a result of interpolating pixelsignals adjacent to each other in the vertical direction is used.Accordingly, the spatial resolution is decreased as compared to whenpixel signals generated from four PG signals respectively havingdifferent phases are used during two frames. Psuedo 4-tap pixelarchitecture, as described in embodiments herein, may use differentmodulation signals for different row pixels. While faster dataacquisition may reduce motion artifacts, depth calculation with twovertical pixels may lower depth resolution. Reflected signal offsetvalues may be used for similarity weights to improve depth resolution.Experimental results may show that edge artifact is improved in thevertical direction. Embodiments of the pseudo 4-tap architecture andexperimental results may be shown in a 2013 IEEE InternationalConference on Consumer Electronics (ICCE) paper titled, “InterpolationMethod for ToF Depth Sensor with Psuedo 4-tap Pixel Architecture,” byTae-Chan Kim, Kwanghyuk Bae, Kyu-Min Kyung and Shung Han Cho, which isincorporated herein by reference in its entirety.

Similarly, when image data is formed using pixel signals only generatedfor the second frame, spatial resolution decreases in the horizontaldirection.

An image signal processor (ISP) 39 generates new image data by mixingthe image data of the first frame and the image data of the secondframe, which are respectively generated based on the first and secondclock signals CLK1 and CLK2 respectively having different frequencies(i.e., a multi-frequency clock signal), thereby preventing a repeateddistance, i.e., an error occurring in depth information due to the limitto the maximum measurement distance. As a result, the depth informationunrestricted by the maximum measurement distance is obtained.

In other embodiments, the maximum measurement distance may be increasedusing at least one more clock signal having a frequency different fromthe frequencies of the first and second clock signals CLK1 and CLK2.

The ISP 39 also generate the new image data by mixing the image data ofthe first frame, which is generated when the same PG signal is appliedto each row, with the image data of the second frame, which is generatedwhen the same PG signal is applied to each column, so that the depthinformation whose spatial resolution is not decreased in the horizontalor vertical direction is obtained. At this time, image data of a totalof four frames are required to obtain image data generated based on twoclock signals respectively having different frequencies or to obtainimage data generated using different PG gate signals applied to rows orcolumns based on a single clock signal in the pixel array 22 includingthe 2-tap depth pixels 23. However, when the pseudo 4-tap scheme isused, image data of just two frames are required. As a result, motionlagging (i.e., an error occurring in the depth information when an imagehas fast movement) occurring when image data generated at differenttimes are mixed is reduced.

The light source driver 30 may generate the clock signal MLS for drivingthe light source 32 under the control of the timing controller 26. Thelight source 32 emits a modulated optical signal EL to a scene 40 inresponse to the clock signal MLS. The scene 40 may be called an objector a target object. The modulated optical signal EL may have differentamplitude according to the operation of the light source driver 30. Thedistance between the scene 40 and the depth sensor 10 may vary.

A light emitting diode (LED), an organic light emitting diode (OLED), anactive-matrix OLED (AMOLED), or a laser diode may be used as the lightsource 32. The clock signal MLS or the modulated optical signal may be asine wave or a square wave.

The light source driver 30 provides the clock signal MLS or informationabout the clock signal MLS to the PG controller 28.

The modulated optical signal EL output from the light source 32 isreflected from the scene 40. When there are different distances Z₁, Z₂,and Z₃ between the depth sensor 10 and the scene 40, a distance Z iscalculated as follows. For instance, when the modulated optical signalEL is Acoswt and a reflected optical signal RL input to the depth pixels23 is A′ cos(ωt+θ)+B′, a phase shift or phase difference θ by TOF isdefined as Equation 1:

θ=2*ω*Z/C=2*(2πf)*Z/C,  (1)

where C is the speed of light.

At this time, the depth pixels 23 means two depth pixels respectivelyincluded in two adjacent rows or columns in the first or second frameand it is assumed that two depth pixels receive the same reflectedoptical signal RL. For instance, the depth pixels 23 may be a depthpixel at the intersection between the first row and the first column anda depth pixel at the intersection between the second row and the firstcolumn in the first frame.

The distance Z from the light source 32 or the pixel array 22 to thescene 40 is calculated using Equation 2:

Z=θ*C/(2*ω)=θ*C/(2*(2πf)).  (2)

The reflected optical signal RL is input to the pixel array 22 throughthe lens module 34. Here, the lens module 34 may include a lens and aninfrared pass filter.

The depth sensor 10 includes a plurality of light sources arranged incircle around the lens module 34, but only one light source 32 isillustrated in FIG. 1 for clarity of the description.

The reflected optical signal RL input to the pixel array 22 through thelens module 34 may be demodulated through a single sampling operation. A2-tap depth pixel basically requires two sampling operations. However,according to the embodiments of the inventive concept, pixel signals A0,A1, A2, and A3 are generated (or detected) by two depth pixelsrespectively included in adjacent rows or columns in the first or secondframe, and therefore, only one sampling operation is required. Asampling operation is generating (or detecting) the pixel signal A0, A1,A2, or A3 from the reflected optical signal RL. The pixel signals A0,A1, A2, and A3 will be described in detail later.

The phase shift θ between the modulated optical signal EL and thereflected optical signal RL may also be defined as Equation 3:

$\begin{matrix}{\theta = {{\arctan \left( \frac{{A\; 3} - {A\; 1}}{{A\; 2} - {A\; 0}} \right)}.}} & (3)\end{matrix}$

Amplitude A of the reflected optical signal RL may be defined asEquation 4:

$\begin{matrix}{A = {\frac{\sqrt{\left( {{A\; 3} - {A\; 1}} \right)^{2} - \left( {{A\; 0} - {A\; 2}} \right)^{2}}}{2}.}} & (4)\end{matrix}$

The amplitude A of the reflected optical signal RL is determined byamplitude of the modulated optical signal EL. An offset B of thereflected optical signal RL may be defined as Equation 5:

$\begin{matrix}{B = {\frac{{A\; 0} + {A\; 2}}{2} = {\frac{{A\; 1} + {A\; 3}}{2}.}}} & (5)\end{matrix}$

Each of the depth pixels 23 includes a single photo diode PD, twotransfer transistors TX1 and TX2, tow reset transistors RX1 and RX2, twodrive transistors DX1 and DX2, and two select transistors SX1 and SX2.In other embodiments, at least one of the reset transistors RX1 and RX2,the drive transistors DX1 and DX2, and the select transistors SX1 andSX2 may be omitted.

In operation of each depth pixel 23, the photo diode PD generatesphotocharge varying with the intensity of the reflected optical signalRL. The first transfer transistor TX1 may transmit the photocharge to afirst floating diffusion node FD1 according to the first PG signal PGS1or the second PG signal PGS2 output from the PG controller 28. Thesecond transfer transistor TX2 may transmit the photocharge to a secondfloating diffusion node FD2 according to the third PG signal PGS3 or thefourth PG signal PGS4 output from the PG controller 28.

In detail, in a period while a PG signal applied to each of the firstand second transfer transistors TX1 and TX2 among the first throughfourth PG signals PGS1 through PGS4 is at a high level (e.g., 3.3 V),each transfer transistor TX1 or TX2 may transmit the photocharge to acorresponding one of the floating diffusion nodes FD1 and FD2.Contrarily, in a period while a PG signal applied to each of the firstand second transfer transistors TX1 and TX2 among the first throughfourth PG signals PGS1 through PGS4 is at a low level (e.g., 0 V), eachtransfer transistor TX1 or TX2 may not transmit the photocharge to acorresponding one of the floating diffusion nodes FD1 and FD2.

The first drive transistor DX1 may amplify and transmit the photochargeto the first select transistor SX1 according to a potential of thephotocharge accumulated at the first floating diffusion node FD1. Thesecond drive transistor DX2 may amplify and transmit the photocharge tothe second select transistor SX2 according to a potential of thephotocharge accumulated at the second floating diffusion node FD2.

A drain terminal of the first select transistor SX1 is connected with asource terminal of the first drive transistor DX1 and may output a pixelsignal to a first column line COL1 in response to a first selectioncontrol signal SEL1 output from the PG controller 28. A drain terminalof the second select transistor SX2 is connected with a source terminalof the second drive transistor DX2 and may output a pixel signal to asecond column line COL2 in response to a second selection control signalSEL2 output from the PG controller 28.

The first reset transistor RX1 may reset the first floating diffusionnode FD1 to VDD according to a first reset signal RS1 output from the PGcontroller 28. The second reset transistor RX2 may reset the secondfloating diffusion node FD2 to VDD according to a second reset signalRS2 output from the PG controller 28.

In other embodiments, the row driver 24 may generate a plurality of thecontrol signals RS1, RS2, SEL1, and SEL2 applied to the depth pixels 23under the control of the timing controller 26.

Each depth pixel 23 accumulates photocharge for a predetermined time,e.g., an integration time and outputs the pixel signals A0, A1, A2, andA3 generated according to the accumulation result. Referring to FIG. 1,under the control of the timing controller 26, a correlated doublesampling (CDS)/analog-to-digital converter (ADC) circuit 36 performs CDSand analog-to-digital conversion on the pixel signals A0, A1, A2, and A3output from the depth pixel 23 and outputs digital pixel signals. Thedepth sensor 10 illustrated in FIG. 1 may also include active loadcircuits (not shown) that transmit a pixel signal output from each of aplurality of column lines in the pixel array 22 to the CDS/ADC circuit36.

A memory 38 may be implemented as a buffer. The memory 38 receives andstores the digital pixel signals output from the CDS/ADC circuit 36.

The depth sensor 10 may also include the ISP 39. The ISP 39 may processthe pixel signals A0, A1, A2, and A3 output from the memory 38 and maycalculate distance information or depth information. The ISP 39 may beimplemented in the semiconductor chip 20. In other embodiments, the ISP39 may be implemented in the outside the semiconductor chip 20 or theoutside of the depth sensor 10.

FIG. 7 is a diagram for explaining a method of interpolating pixelsignals output from the pixel array 22 illustrated in FIG. 1 using thepseudo 4-tap scheme according to some embodiments of the inventiveconcept. FIG. 8 is a diagram illustrating reflected optical signals RLsinput to some of pixels illustrated in FIG. 7. FIGS. 9A and 9B arediagrams illustrating a weight function used in interpolating the pixelsignals.

Referring to FIGS. 1 through 9B, FIG. 7 shows 25 pixels arranged in amatrix. Although only 25 pixels are illustrated in the currentembodiments for clarity of the description, the scope of the inventiveconcept is not restricted to the current embodiments. A pixel P(i,j) isa pixel positioned at the intersection between an i-th row and a j-thcolumn.

As illustrated in FIGS. 3 and 4, the first PG signal PGS1 and the thirdPG signal PGS3 are applied to the (i−2)-th row, the i-th row, and the(i+2)-th row and the second PG signal PGS2 and the fourth PG signal PGS4are applied to the (i−1)-th row and the (i+1)-th row in the first frame.The first PG signal PGS1 and the third PG signal PGS3 are applied to the(j−2)-th column, the j-th column, and the (j+2)-th column and the secondPG signal PGS2 and the fourth PG signal PGS4 are applied to the (j−1)-thcolumn and the (j+1)-th column in the second frame.

As described above with reference to FIGS. 5 and 6, interpolation isperformed using the pixel signals A0 and A2 or A1 and A3 generated by acurrent depth pixel 23 in the first or second frame and pixel signals ofadjacent depth pixels 23 of the current depth pixel 23 and image datarespectively generated in the two frames are mixed with each other, sothat image data whose spatial resolution is not decreased in thevertical or horizontal direction is obtained. The ISP 39 may interpolatea pixel signal of the pixel P(i,j) using pixel signals of pixelsincluded in the (i−1)-th row and pixel signals of pixels included in the(i+1)-th row in the 5×5 matrix during the first frame because the(i−1)-th row and the (i+1)-th row receive PG signals having a differentphase from a PG signal applied to the pixel P(i,j). Similarly, the ISP39 may interpolate the pixel signal of the pixel P(i,j) using pixelsignals of pixels included in the (j−1)-th column and pixel signals ofpixels included in the (j+1)-th column in the 5×5 matrix during thesecond frame because the (j−1)-th column and the (j+1)-th column receivePG signals having a different phase from a PG signal applied to thepixel P(i,j).

In other words, when pixels receiving the first PG signal PGS1 and thethird PG signal PGS3 in each frame are defined as first pixels andpixels receiving the second PG signal PGS2 and the fourth PG signal PGS4in each frame are defined as second pixels, a pixel signal of a firstpixel may be interpolated using pixel signals of adjacent second pixelsand a pixel signal of a second pixel may be interpolated using pixelsignals of adjacent first pixels.

For clarity of the description, a case where the ISP 39 interpolates thepixel signal of the pixel P(i,j) using a pixel signal of a pixelP(i−1,j) and a pixel signal of a pixel P(i+1,j) in the first frame willbe described. However, the scope of the inventive concept is notrestricted to this case and the number or positions of pixels used forinterpolation are not restricted.

The pixel signal of the pixel P(i,j) includes P_(i,j,0) and P_(i,j,π)respectively generated based on the first PG signal PGS1 and the thirdPG signal PGS3. P_(i,j,0) and P_(i,k,π) are respectively defined asEquations 6 and 7:

P _(i,j,0)=AM1 cos θ1+B1,  (6)

and

P _(i,j,π)=AM1 cos(θ1+π)+B1=−AM1 cos θ1+B1,  (7)

where AM1, θ1, and B1 are the amplitude, phase shift, and offset,respectively, of a reflected optical signal RL(i,j).

The pixel signal of the pixel P(i−1,j) includes P_(i−1,j,π/2) andP_(i−1,j,3π/2) respectively generated based on the first PG signal PGS1and the third PG signal PGS3 and the pixel signal of the pixel P(i+1,j)includes P_(i+1,j,π/2) and P_(i+1,j,3π/2) respectively generated basedon the first PG signal PGS1 and the third PG signal PGS3. P_(i−1,j,π/2),P_(i−1,j,3π/2), P_(i+1,j,π/2), and P_(i+1,j,3π/2) are respectivelydefined as Equations 8 through 11:

$\begin{matrix}{{P_{{i - 1},j,\frac{\pi}{2}} = {{{{AM}\; 2{\cos \left( {{\theta 2} + \frac{\pi}{2}} \right)}} + {B\; 2}} = {{{AM}\; 2\sin \; {\theta 2}} + {B\; 2}}}},} & (8) \\{{P_{{i - 1},j,\frac{3\pi}{2}} = {{{{AM}\; 2{\cos \left( {{\theta 2} + \frac{3\pi}{2}} \right)}} + {B\; 2}} = {{{- {AM}}\; 2\sin \; {\theta 2}} + {B\; 2}}}},} & (9) \\{{P_{{i + 1},j,\frac{\pi}{2}} = {{{{AM}\; 3{\cos \left( {{\theta 3} + \frac{\pi}{2}} \right)}} + {B\; 3}} = {{{AM}\; 3\sin \; {\theta 3}} + {B\; 3}}}},{and}} & (10) \\{{P_{{i + 1},j,\frac{3\pi}{2}} = {{{{AM}\; 3{\cos \left( {{\theta 3} + \frac{3\pi}{2}} \right)}} + {B\; 3}} = {{{- {AM}}\; 3\sin \; {\theta 3}} + {B\; 3}}}},} & (11)\end{matrix}$

where AM2, θ2, and B2 are the amplitude, phase shift, and offset,respectively, of a reflected optical signal RL(i−1,j) and AM3, θ3, andB3 are the amplitude, phase shift, and offset, respectively, of areflected optical signal RL(i+1,j).

The offsets B2 and B3 are easily obtained by averaging P_(i−1,j,π/2) andP_(i−1,j,3π/2) and averaging P_(i+1,j,π/2) and P_(i+1,j,3π/2). It isassumed that the offset sets of reflected optical signals input topixels on the upper left of a border BO in FIG. 7 are the same as oneanother and that the offsets of reflected optical signals input topixels on the lower right of the border BO are the same as one anotherbut are lower than the offsets of the reflected optical signals input tothe pixels on the upper left of the border BO.

The offset of each reflected optical signal is related with externallight apart from modulated light emitted by the light source 32 and withon/off of the light source 32. In other words, an image based on onlyoffset of each reflected optical signal in the entire pixel array 22includes position information of an object's shape formed according toexternal light conditions, as if it is photographed in black and white.This means that an object sensed by the pixels on the upper left of theborder BO may be different from that sensed by the pixels on the lowerright of the border BO. Therefore, when interpolation is performed usingthe pseudo 4-tap scheme according to embodiments of the inventiveconcept, a weight is applied to each of the pixel signals used for theinterpolation according to the offset of a reflected optical signal.

According to the pseudo 4-tap scheme, the phase shift θ of the reflectedoptical signal RL(i,j) input to the pixel P(i,j) may be calculated usingEquation 12:

$\begin{matrix}{{\theta = {\arctan \left( \frac{P_{3{\pi/2}} - P_{\pi/2}}{P_{i,j,\pi} - P_{i,j,0}} \right)}},} & (12)\end{matrix}$

where P_(3π/2) is a result of interpolating P_(i−1,j,π/2) andP_(i−1,j,3π/2) and P_(π/2) is a result of interpolating P_(i+1,j,π/2)and P_(i+1,j,3π/2). P_(3π/2) and P_(π/2) may be respectively calculatedusing Equations 13 and 14:

$\begin{matrix}{{{P_{\frac{3\pi}{2}} = {\frac{1}{K}{\sum\limits_{m,n,\frac{3\pi}{2}}^{P}{{f\left( \sqrt{\left( {m - i} \right)^{2} + \left( {n - j} \right)^{2}} \right)}{g\left( {B^{\prime} - {B\; 1}} \right)}}}}},{and}}{{P_{\frac{\pi}{2}} = {\frac{1}{K}{\sum\limits_{m,n,\frac{\pi}{2}}^{P}{{f\left( \sqrt{\left( {m - i} \right)^{2} + \left( {n - j} \right)^{2}} \right)}{g\left( {B^{\prime} - {B\; 1}} \right)}}}}},}} & (13)\end{matrix}$

where K is the number of pixel signals used for the interpolation, “f”and “g” are weight functions, and P_(m,n,3π/2) and P_(m,n,π/2) are pixelsignals used for the interpolation. The independent variable of thefunction “f” indicates the relative position of a depth pixelcorresponding to P_(m,n, 3π/2) and P_(m,n,π/2) and the pixel P(i,j). Theindependent variable of the function “g” indicates the differencebetween an offset B′ (e.g., B2 or B3) of the reflected optical signal(e.g., RL(i−1,j) or RL(i+1,j)) input to the depth pixel corresponding toP_(m,n,3π/2) and P_(m,n,π/2) and the offset B of the reflected opticalsignal RL(i,j) input to the pixel P(i,j).

In the current case, P_(3π/2) and P_(π/2) may be respectively calculatedusing Equations 15 and 16:

$\begin{matrix}{{P_{\frac{3\pi}{2}} = {\frac{1}{2}\left( {{P_{{i - 1},j,\frac{3\pi}{2}}{f(1)}{g\left( {{B\; 2} - {B\; 1}} \right)}} + {P_{{i + 1},j,\frac{3\pi}{2}}f(1){g\left( {{B\; 3} - {B\; 1}} \right)}}} \right)}},} & (15) \\{and} & \; \\{P_{\frac{\pi}{2}} = {\frac{1}{2}{\left( {{P_{{i - 1},j,\frac{\pi}{2}}{f(1)}{g\left( {{B\; 2} - {B\; 1}} \right)}} + {P_{{i + 1},j,\frac{\pi}{2}}f(1){g\left( {{B\; 3} - {B\; 1}} \right)}}} \right).}}} & (16)\end{matrix}$

Referring to FIG. 9A, it is assumed that the weight is 0.3 when theindependent variable of the function “f” is 1. Referring to FIG. 9B, itis assumed that the weight is 1 when the independent variable of thefunction “g” is 0 and the weight is 0.2 when the independent variable ofthe function “g” is B2−B1. For instance, the function “f” and thefunction “g” may be Gaussian functions.

The pixel P(i−1,j) and the pixel P(i+1,j) have the same weights (e.g.;f(1) and f(1)) according to the relative position therebetween but havedifferent weights (e.g., g(B2−B1) and g(B3−B1) according to an offsetdifference, i.e., position information of an object's shape formedaccording to external light conditions. That is, g(B2−B1)=0, and sincethe pixel P(i+1,j) is positioned at the same side as the pixel P(i,j) onthe basis of the border BO, g(B3−B1)=g(0)=1. In other words, when thepixel signal of the pixel P(i,j) is interpolated, a weight for the pixelsignal of the pixel P(i+1,j) is higher than that for the pixel signal ofthe pixel P(i−1,j). Accordingly, the interpolation is performed so thatthe object's shape is closer to an actual shape, and therefore, moreaccurate distance computation is possible.

According to the current embodiments of the inventive concept, the depthsensor 10 uses an offset of a pixel signal during the interpolation ofthe pixel signal, thereby enabling accurate distance information to beobtained.

FIG. 10 is a flowchart of a method of operating the depth sensor 10illustrated in FIG. 1 according to some embodiments of the inventiveconcept. FIG. 11 is a detailed flowchart of an operation ofinterpolating pixel signals in the method illustrated in FIG. 10.

Referring to FIGS. 1 through 11, the PG signal generator 53 may generatethe first through fourth PG signals PGS1 through PGS4 based on the firstclock signal CLK1 or the second clock signal CLK2 in operation 5100. Therow and column selector 54 may apply the first through fourth PG signalsPGS1 through PGS4 to the rows of the pixel array 22 in a first frameaccording to the control of the frame index 52 in operation S110. Therow and column selector 54 may apply the first through fourth PG signalsPGS1 through PGS4 to the columns of the pixel array 22 in a second frameaccording to the control of the frame index 52 in operation S120.

The ISP 39 may interpolate a pixel signal output from each of the pixels23 included in the pixel array 22 and generate image data based oninterpolated pixel signals in operation S130. Operation S130 may includeoperations S132 through S138.

The ISP 39 may interpolate a pixel signal of a first pixel (e.g., thepixel P(i,j)) included in a first row (e.g., the i-th row in FIG. 7)based on a pixel signal of at least one second pixel (e.g., the pixelP(i−1,j)) included in a second row (e.g., the (i−1)-th row in FIG. 7) inthe first frame in operation S132. At this time, a weight (e.g., thecalculated value of a Gaussian function) corresponding to an offset(e.g., B2) is applied to the pixel signal of the at least one secondpixel (e.g., the pixel P(i−1,j)).

The ISP 39 may interpolate the pixel signal of the second pixel (e.g.,the pixel P(i−1,j)) included in the second row (e.g., the (i−1)-th rowin FIG. 7) based on a pixel signal of at least one first pixel (e.g.,the pixel P(i,j)) included in the first row (e.g., the i-th row in FIG.7) in the first frame in operation 5134. At this time, a weight (e.g.,the calculated value of a Gaussian function) corresponding to an offset(e.g., B1) is applied to the pixel signal of the at least one firstpixel (e.g., the pixel P(i,j)).

The ISP 39 may interpolate a pixel signal of a first pixel (e.g., thepixel P(i,j)) included in a first column (e.g., the j-th column in FIG.7) based on a pixel signal of at least one second pixel (e.g., the pixelP(i,j−1)) included in a second column (e.g., the (j−1)-th column in FIG.7) in the second frame in operation 5136. At this time, a weight (e.g.,the calculated value of a Gaussian function) corresponding to an offset(e.g., B2) is applied to the pixel signal of the at least one secondpixel (e.g., the pixel P(i,j−1)).

The ISP 39 may interpolate the pixel signal of the second pixel (e.g.,the pixel P(i,j−1)) included in the second column (e.g., the (j−1)-thcolumn in FIG. 7) based on a pixel signal of at least one first pixel(e.g., the pixel P(i,j)) included in the first column (e.g., the j-thcolumn in FIG. 7) in the second frame in operation S138. At this time, aweight (e.g., the calculated value of a Gaussian function) correspondingto an offset (e.g., B1) is applied to the pixel signal of the at leastone first pixel (e.g., the pixel P(i,j)).

The present general inventive concept can also be embodied ascomputer-readable codes on a computer-readable medium. Thecomputer-readable recording medium is any data storage device that canstore data as a program which can be thereafter read by a computersystem. Examples of the computer-readable recording medium includeread-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetictapes, floppy disks, and optical data storage devices.

The computer-readable recording medium can also be distributed overnetwork coupled computer systems so that the computer-readable code isstored and executed in a distributed fashion. Also, functional programs,codes, and code segments to accomplish the present general inventiveconcept can be easily construed by programmers.

As described above, according to some embodiments of the inventiveconcept, a depth sensor differently applies a multi-frequency PG signalfor each frame, thereby providing image data for which spatialresolution is not decreased, motion lagging is decreased and ameasurable distance is increased. In addition, the depth sensor uses anoffset of a pixel signal when interpolating the pixel signal, therebyobtaining accurate distance information.

While the inventive concept has been particularly shown and describedwith reference to exemplary embodiments thereof, it will be understoodby those of ordinary skill in the art that various changes in forms anddetails may be made therein without departing from the spirit and scopeof the inventive concept as defined by the following claims.

What is claimed is:
 1. A method of operating a depth sensor, the methodcomprising: generating a first photo gate signal and second throughfourth photo gate signals respectively having 90-, 180- and 270-degreephase differences from the first photo gate signal; applying the firstphoto gate signal and the third photo gate signal to a first row of apixel array and the second photo gate signal and the fourth photo gatesignal to a second row adjacent to the first row in a first frame usinga first clock signal; and applying the first photo gate signal and thethird photo gate signal to a first column of the pixel array and thesecond photo gate signal and the fourth photo gate signal to a secondcolumn adjacent to the first column in a second frame using a secondclock signal, wherein a frequency of the first clock signal is differentfrom a frequency of the second clock signal.
 2. The method of claim 1,further comprising: interpolating pixel signals output from respectivepixels comprised in the pixel array; and generating image data based oninterpolated pixel signals.
 3. The method of claim 2, wherein theinterpolating the pixel signals comprises: interpolating a pixel signalof a pixel in the first row based on a pixel signal of at least onepixel in the second row in the first frame; and interpolating a pixelsignal of a pixel in the second row based on a pixel signal of at leastone pixel in the first row in the first frame.
 4. The method of claim 3,wherein the interpolating the pixel signal of the pixel in the first rowbased on the pixel signal of the at least one pixel in the second rowcomprises applying a weight corresponding to an offset of the at leastone pixel in the second row to the pixel signal of the at least onepixel in the second row; and the interpolating the pixel signal of thepixel in the second row based on the pixel signal of the at least onepixel in the first row comprises applying a weight corresponding to anoffset of the at least one pixel in the first row to the pixel signal ofthe at least one pixel in the first row.
 5. The method of claim 4,wherein the weight is calculated based on a Gaussian function.
 6. Themethod of claim 3, wherein the interpolating the pixel signals furthercomprises: interpolating a pixel signal of a pixel in the first columnbased on a pixel signal of at least one pixel in the second column inthe second frame; and interpolating a pixel signal of a pixel in thesecond column based on a pixel signal of at least one pixel in the firstcolumn in the second frame.
 7. The method of claim 6, wherein theinterpolating the pixel signal of the pixel in the first column based onthe pixel signal of the at least one pixel in the second columncomprises applying a weight corresponding to an offset of the at leastone pixel in the second column to the pixel signal of the at least onepixel in the second column; and the interpolating the pixel signal ofthe pixel in the second column based on the pixel signal of the at leastone pixel in the first column comprises applying a weight correspondingto an offset of the at least one pixel in the first column to the pixelsignal of the at least one pixel in the first column.
 8. The method ofclaim 7, wherein the weight is calculated based on a Gaussian function.9. A depth sensor comprising: a photo gate signal generator configuredto generate a first photo gate signal and second through fourth photogate signals respectively having 90-, 180- and 270-degree phasedifferences from the first photo gate signal; and a row and columnselector configured to apply the first through fourth photo gate signalsto rows or columns of a pixel array, the row and column selector beingconfigured to apply the first photo gate signal and the third photo gatesignal to a first row of the pixel array and the second photo gatesignal and the fourth photo gate signal to a second row adjacent to thefirst row in a first frame using a first clock signal and beingconfigured to apply the first photo gate signal and the third photo gatesignal to a first column of the pixel array and the second photo gatesignal and the fourth photo gate signal to a second column adjacent tothe first column in a second frame using a second clock signal, whereina frequency of the first clock signal is different from a frequency ofthe second clock signal.
 10. The depth sensor of claim 9, furthercomprising an image signal processor configured to interpolate pixelsignals output from respective pixels comprised in the pixel array andto generate image data based on interpolated pixel signals.
 11. Thedepth sensor of claim 10, wherein the image signal processor isconfigured to interpolate a pixel signal of a pixel in the first rowbased on a pixel signal of at least one pixel in the second row in thefirst frame and is configured to interpolate the pixel signal of a pixelin the second row based on a pixel signal of at least one pixel in thefirst row in the first frame.
 12. The depth sensor of claim 11, whereinthe image signal processor is configured to apply a weight correspondingto an offset of the at least one pixel in the second row to the pixelsignal of the at least one pixel in the second row when interpolatingthe pixel signal of the pixel in the first row based on the pixel signalof the at least one pixel in the second row and is configured to apply aweight corresponding to an offset of the at least one pixel in the firstrow to the pixel signal of the at least one pixel in the first row wheninterpolating the pixel signal of the pixel in the second row based onthe pixel signal of the at least one pixel in the first row.
 13. Thedepth sensor of claim 10, wherein the image signal processor isconfigured to interpolate a pixel signal of a pixel in the first columnbased on a pixel signal of at least one pixel in the second column inthe second frame and is configured to interpolate a pixel signal of apixel in the second column based on a pixel signal of at least one pixelin the first column in the second frame.
 14. The depth sensor of claim13, wherein the image signal processor is configured to apply a weightcorresponding to an offset of the at least one pixel in the secondcolumn to the pixel signal of the at least one pixel in the secondcolumn when interpolating the pixel signal of the pixel in the firstcolumn based on the pixel signal of the at least one pixel in the secondcolumn and is configured to apply a weight corresponding to an offset ofthe at least one pixel in the first column to the pixel signal of the atleast one pixel in the first column when interpolating the pixel signalof the pixel in the second column based on the pixel signal of the atleast one pixel in the first column.
 15. A method of operating a depthsensor, the method comprising: interpolating pixel signals output frompixels of first and second adjacent rows of a pixel array in a firstframe, wherein the pixels of the first and second adjacent rows of thepixel array generate the pixel signals in response to an optical signalreflected from a scene; interpolating pixel signals output from pixelsof first and second adjacent columns of the pixel array in a secondframe, wherein the pixels of the first and second adjacent columns ofthe pixel array generate the pixel signals in response to an opticalsignal reflected from the scene, and wherein the pixel signals of thefirst frame are generated using a first clock signal and the pixelsignals of the second frame are generated using a second clock signaldifferent than the first clock signal; and generating image data basedon the interpolated pixel signals, wherein the pixel signals of thefirst row are generated by a first pair of photo gate signals out ofphase with each other and the pixel signals of the second row aregenerated by a second pair of photo gate signals out of phase with eachother, and wherein the pixel signals of the first column are generatedby the first pair of photo gate signals and the pixel signals of thesecond column are generated by the second pair of photo gate signals.16. The method of claim 15, wherein the first pair of photo gate signalscomprise first and third photo gate signals, and the second pair ofphoto gate signals comprise second and fourth photo gate signals, andwherein the second, third and fourth photo gate signals respectivelyhave 90-, 180- and 270-degree phase differences from the first photogate signal.
 17. The method of claim 15, wherein the interpolating thepixel signals comprises: interpolating a pixel signal of a first pixelin the first row based on a pixel signal of a second pixel in the secondrow; interpolating the pixel signal of the second pixel in the secondrow based on the pixel signal of the first pixel in the first row;interpolating a pixel signal of a first pixel in the first column basedon a pixel signal of a second pixel in the second column; andinterpolating the pixel signal of the second pixel in the second columnbased on the pixel signal of the first pixel in the first column. 18.The method of claim 17, wherein the interpolating the pixel signal ofthe first pixel in the first row based on the pixel signal of the secondpixel in the second row comprises applying a weight corresponding to areflected signal offset of the second pixel to the pixel signal of thesecond pixel; the interpolating the pixel signal of the second pixel inthe second row based on the pixel signal of the first pixel in the firstrow comprises applying a weight corresponding to a reflected signaloffset of the first pixel to the pixel signal of the first pixel; theinterpolating the pixel signal of the first pixel in the first columnbased on the pixel signal of the second pixel in the second columncomprises applying a weight corresponding to a reflected signal offsetof the second pixel to the pixel signal of the second pixel; and theinterpolating the pixel signal of the second pixel in the second columnbased on the pixel signal of the first pixel in the first columncomprises applying a weight corresponding to a reflected signal offsetof the first pixel to the pixel signal of the first pixel.
 19. Themethod of claim 15, wherein a frequency of the first clock signal isdifferent from a frequency of the second clock signal.